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ABSTRACT 


Flight control and procedural task skill degradation, and the 
effectiveness of retraining methods were evaluated for a simulated 
space vehicle approach and landing under instrument and visual flight 
conditions. Fifteen experienced pilots were trained and then tested 
after 4 months either without the benefits of practice or with static 
rehearsal (checklists and briefings), dynamic rehearsal (briefings and 
videc taped flight presentations), or with dynamic warmup practice 
(closed loop simulator practice). Performance on both the flight 
control and procedure tasks degraded significantly after 4 months. 

The rehearsal methods effectively countered procedure task skill 
degradation, while dynamic rehearsal or a combination of static 
rehearsal and dynamic warmup practice was required for the flight 
control tasks. The quality of the retraining methods appeared to be 
primarily dependent on the efficiency of visual cue reinforcement. 
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FOREWORD 

This report summarizes an experimental study accomplished as the 
third part of a program designed to investigate the degradation of 
learned skills as applicable to spaceflight tasks. The research 
reported here was begun in July 1971 and was completed in August 1972 
for the NASA Manned Spacecraft Center under Contract NAS9-1 0962 . The 
study was initiated by Dr. William E. Fedderson, Chief of the Behavioral 
Laboratory, Biomedical Laboratories Division. Dr. Fedderson was the 
NASA Project Monitor throughout the study. 

The Boeing Program Manager was Dr. George D. Greer, Jr. and the 
Principal Investigator was Dr. Thomas E. Sitterley. The authors grate- 
fully acknowledge the extensive assistance of Mr. Gale M. Rhoades who 
contributed to simulator modification, operation, and data reduction, 
to Messrs. David Tubb, Louis Hough and Douglas Berg for their contri- 
bution in math modeling, computer programming and flight simulator 
operations, and to Mr. Allen Fukushima for his engineering assistance 
in terrain model and visual systems modification and operation. 

The first part of this investigation of degradation of learned 
skills was covered in Report D1 80- 1 5080-1 , Degradation of Learned Skills - 
A review and Annotated Bibliography. The second part was covered in 
Report D 1 80-1 5081-1 , Degradation of Learned Skills - Effectiveness of 
Practice Methods on Simulated Space Flight Skill Retention. 
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1. INTRODUCTION 

The ability of pilots to maintain flight control skills over periods 
of operational inactivity has long been of considerable concern. 
Practical experience and reviews of many years of research (Naylor and 
Briggs, 1961; Gardlin.and Sitterley, 1972) graphically demonstrate the 
susceptibility of skilled task performance to degrade with the passage 
of time. However, not all of the data or experiences fit this gener- 
alization. Much of the conflict apparently occurs because many 
studies or observations, presumably directed toward the same question, 
look at completely different tasks, performed by dissimilar subject 
populations, and measured against desparate performance criteria. 

Particularly critical are the task characteri sti cs and specification 
of performance criteria. When carefully defined and compared, an 
apparent superiority in skill retention is found for continuous 
control tasks as opposed to procedural tasks. Similarly, the relative 
benefits of the same amount and type of practice are generally greater 
for procedural tasks as compared to continuous control tasks. Naylor 
and Briggs (1961) suggested that the primary difference between the 
two types; was largely a question of organization. Typically the 
procedural task is held together with less spatial or temporal con- 
tinuity whereas each element of the continuous control task relates to 
the previous element and suggests or reinforces the succeeding element. 

While useful from the standpoint of task description, the procedure 
task/continuous control task dichotomy can lead not only to an 
incorrect prediction of the retention of flight skills but also an 
inappropriate specification of retraining methods. As pointed out 
by Sitterley and Berge (1972), piloting an aircraft, while primarily 
psychomotor in nature, requires a significant cognitive contribution 
in terms of information integration and decision making. Cognitive, 
discrete, and continuous control task elements are represented in the 
flight control task and these same elements are found in varying 
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degree as part of procedural task performance. The Sitterley and Berge 
study graphically demonstrated the requirement to understand the 
relationship of these elements to the total task in terms of the 
defined measures of performance. 

In that study, subjects were trained to manually control a simulated 
space vehicle from launch through orbit insertion. Flight performance 
was evaluated by measures of integrated pitch and altitude error from 
the desired flight profile and discrete measures of altitude and rate 
errors at orbit insertion. Throughout the flight, emergency procedure 
performance was measured in terms of time and error. The effects of 
no practice, rehersal practice, and warmup practice on skill retention 
were factorially evaluated over retention intervals from 1 to 6 months. 

The contention that the rate and magnitude of performance degradation 
was not only a function of time and type of training, but also closely 
related to the performance measurements, was substantiated. Inter- 
preted in relation to each other, the performance measures indicated 
that the procedure task degraded more consistently and to a greater 
degree than the flight control task, and while static rehersal 
effectively countered procedural degradation, some form of dynamic 
warmup appeared necessary for flight control skill retention. How- 
ever, for both tasks, the various performance measures sampled task 
elements across the continuum from cognitive decision making to 
discrete and continuous psychomotor control. As such, each measure 
taken by itself could have resulted in different conclusions as a 
function of the task element sampled. 

While the Sitterley and Berge study assessed flight control and proce- 
dural skills, it did not address one of the critical elements of pilot 
performance: far-field vision and perceptual cues. The requirement 

for far-field (out-the-window) visual perception is an inherent part 
of the landing. Far-field visual perception has been long recognized 
as one of the most critical aspects of airplane flight, and emphasized. 
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of course, in its relation to a successful approach and landing. 
However, surprisingly little is known about when far-field visual 
tasks can be expected to deteriorate beyond the point of acceptability. 

Certainly, the complex perceptual cues and additional burden of 
integrating the visual information with the flight task would suggest 
that the approach and landing task would be subject to greater skill 
degradation than that found by Sitterley and Berge. Further the 
operational tasks should be more complex in terms of the perceptual/ 
motor coordination, timing, and task load. Therefore, even with 
experienced pilots, the previous data would suggest a no-practice 
limit of 3 months or less is required to preserve acceptable flight 
skills. This estimate fits quite closely to common naval aviation 
procedure of requiring carrier landing practice at least once a month 
to preserve the perceptual coordination and timing. 

The next question is how to counter visual flight control skill degra- 
dation. Dynamic closed-loop flight trainers with elaborate visual 
simulation attachments are currently available. The use of these 
trainers in conjunction with actual flight has proven to be a cost 
effective approach for initial pilot training. However, after a pilot 
has reached a high level of proficiency and experience, it is unknown 
if this level of trainer sophistication is required for periodic 
retraining, or if other methods might suffice. It is true that simu- 
lator based training is less expensive than actual flight training, 
and frequently provides the only available method for training. 
Nevertheless, training simulators are still very expensive to obtain 
and operate. Further, many circumstances of cost, geographic, and 
space/weight/power restrictions limit the use or availability of 
simulators to provide flight skill practice during extended periods 
of flight inactivity. 


Purpose 

Using a visual flight simulator, experienced pilots, and an opera- 
tionally oriented flight task, the purpose of this study was: a) to 
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quantify the magnitude of degradation of flight control tasks which 
involve the use of far-point (out-the-window) visual cues; b) to 
investigate the retraining effectiveness of non-simulator static 
rehersal and open-loop dynamic display practice in comparison to 
dynamic closed-loop warmup practice; and c) to evaluate these skill 
retention methods for normal and emergency mode procedures under 
conditions of high task load. 
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2. METHOD 

Experienced pilots were trained to fly a simulated spacecraft of the 
H - 33 Space Shuttle orbiter configuration through an approach and 
landing. Flight control and procedural performance was measured at 
the end of training and again at the end of four months with and with- 
out the benefits of practice. 


Subjects 

Fifteen experienced pilots currently not flying were used in this study. 
In order to reduce the amount of initial training required and to 
increase the uniformity and representativeness of the subject popu- 
lation, the pilots were obtained from Boeing engineering and technical 
staff groups. The group was a mixture of flight test engineers, 
control systems personnel, training requirements staff, and crew 
systems engineers. 

The subject population was required to meet the following criteria: 

(1) previous formal flight training and experience as a pilot; (2) 
commitment to no flight activities during the test period; (3) vision, 
20/30, corrected; and (4) under 55 years of age. 

The average age of the pilot population was 44.4 years with a range of 
from 33 to 50. The experience level of the subjects averaged 5,300 
pilot hours with a range of from 900 to 12,600 hours. They averaged 
1,250 instrument hours with a range of from 100 to 5,000 hours. The 
pilots averaged 5.3 years since their last flight; the most current 
had been flying up to within 6 months before the start of the test; 
one of the group had not flown for 16 years. 

Task Description 

The pilots task was to control the vehicle from an altitude of 31,000 
feet through a descending turn to an approach and landing on a runway. 
Figure 1 depicts a schematic of the basic flight profile which 
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required approximately 6 min. 45 sec. to complete. The detailed 
mission description, approach data, and charts are presented in the 
appendix. Basically, the flight profile assumed that the pilot has 
just made a successful de-orbit and reentry pass through the transition 
stage. 


360 



EDW (5) 


Figure 1: Flight Schematic-Edwards AFB (SIM) Simulation 

Approach, H-33 Orbiter 


The test mission began at 31,400 feet, 15 nautical miles from a 
simulated Edwards TACAN. The approach and landing were made un- 
powered. Ceiling was 10,000 feet, overcast, visibility 15 miles; the 
cloud deck was solid through 35,000 feet. A turning approach descend- 
ing at about 5,000 feet per minute was made to the TACAN using instru- 
ments only (IFR). Energy management was accomplished through 
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judicious use of speed boards at an equivalent airspeed of 240 knots. 
Stabilization on the localizer and glideslope provided a straight-in 
approach to the Edwards runway 12 miles from the TACAN. During this 
portion of the flight, the pilot was required to perform emergency 
procedures to correct a series of malfunctions in the vehicle's flight 
control system (SAS Failure Procedure). 

After crossing the TACAN station, a complete electrical power failure 
occurred. At this time, the pilot was required to perform a corrective 
procedure (Subsystem Scan). During the failure, the vehicle was 
repositioned to one of a standardized set of offsets from the flight 
path. These offsets, presented in random order, permited the evalua- 
tion of the final visual approach performance from a known starting 
point for all pilots. Upon power recovery (in 12 seconds), the pilot 
continued the descent on instruments through 10,000 feet, applying 
corrective control inputs to return the vehicle to the desired flight 
path . 

At this point, the pilot broke out visually and was able to use the 
combination of instruments and external visual environment in 
establishing the required lineup and glideslope. At 8,000 ft, the 
on-board terminal navigation system failed, and the pilot was required 
to perform another subsystem scan corrective procedure. No correction 
of the failure was possible, forcing the pilot to make the remaining 
approach and final touchdown under visual conditions (VFR) with only 
basic vehicle attitude, speed, and altitude information. 

Equipment 

The experimental test was conducted using the simulation facilities of 
the Boeng Aerospace Group in Seattle. The simulation equipment was 
comprised of four major parts: (a) cockpit with associated displays 

and controls, (b) visual simulation system, (c) computer and simulation 
control system, and (d) the procedure task function logic system. The 
equipment and associated computer software was integrated to provide 
a highly realistic simulation of a fully aerodynamic Space Shuttle 
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W^STopy. 


orbiter descent, approach and landing as controlled visually and by 
instruments from a one-man cockpit. 


Cockpit 

A one-man cockpit, configured with all displays and controls required 
to fly the simulated mission was used for both pilot training and 
retention testing. The cockpit, used for general purpose part-task 
simulation studies, was reconfigured for this experiment. No attempt 
was made to duplicate any Space Shuttle cockpit concepts. Figure 2 
shows the general cockpit display/control configuration in relation to 
a simulation pilot. 



Figure 2: Simulation Cockpit with Pilot 
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External out-the-window visual scenes were simulated using a 21-inch 
cathode ray tube. This 1029 line television display was viewed 
through a set of acrylic lenses which produced the visual image at 
optical infinity. This infinity optics system provided a field of 
view of approximately 40 degrees through a centrally located windscreen 

A detailed description of the cockpit, all displays and controls and 
their function and use is described in the Flight Control and 
Procedure Training Package contained in the appendix. Basically, the 
displays included electromechanical and cathode ray tube displays 
for attitude, velocity, altitude, course, and status information. An 
X-20 type, two-axis, sidearm controller provided proportional rate 
commands for pitch and roll. Rudder pedals provided displacement 
commands for yaw. Pitch trim and speedboard commands were provided 
through discrete rate controls. 

Visual Simulation System 

The external environment seen by the pilot through the electro-optical 
windscreen display was produced by the Boeing visual flight simulator. 
This simulation system made use of high resolution television cameras, 
computer controlled to fly over terrain models. The high resolution 
closed circuit television system consisted of cameras mounted on two 
precision 6 degree freedom television camera/servo systems. The 
video signals were fed through a special effects/video mixing control 
to the 1029 line TV monitor in the cockpit. 

The 1 inch vidicon cameras operated with a 1029 line standard. Horizon 
tal resolution was 700 television lines and vertical resolution was 
650 television lines. Scanning linearity was 1.5 percent across the 
field of view. Each camera was mounted on a rail guided carriage and 
gimbal system. The computer controlled carriages and gimbals were 
digitally positioned in front of the two terrain models. Precision 
control was maintained with both positional and velocity feedback 
signals to an accuracy of 0.001 in. in translation and 3 minutes of 
arc in rotation. 
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Two scale relief terrain models were used during the visual portion 
approach. These models provided a realistic view of a modified ap- 
proach to Edwards AFB from an altitude of 10,000 ft to touchdown. 

An arbitrary runway heading of 360° was used, with the runway scaled 
to the dimensions of 200 x 10,000 feet. Figure 3 depicts Model II 
of Edwards AFB and one camera/servo system carriage. This model was 
11 ft x 24 ft (Scale 1:6250) and provided terrain feature representation 
to a vehicle altitude of 175 ft. Figure 4 depicts the camera eye view 
of Model II, approximately 5 miles from the runway threshold. 



Figure 3: Terrain Model II - Edwards AFB, with Camera Stage 

and Lighting Mirror 
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Figure 4: Camera View of Approach, Approximately 5 Miles 

from Threshold 


Model I provided the detailed representation of the runway for pilot's 
eye altitudes of 300 ft to 20 ft (Scale 1:200). This 11 ft by 90 ft 
model is depicted in Figure 5 along with its camera/servo system. 
During pilot training and performance testing, the Model I approach 
lights and adjacent terrain were replaced with dry lake bed features, 
scaled and contrast matched to Model II. 


Figure 5: Terrain Model I - Runway, and Camera/Servo System 



The visual transition between the two models occurred when the vehicle 
passed through an altitude of 300 ft. The landing model camera stage 
was set in motion by the computer while the other camera stage was 
still flying. After the camera was synchronized with the vehicle's 
flight, the visual transition was accomplished by computer controlled 
video fade-in/fade-out of the two TV camera/terrain model systems. 

The visual image transition between the two terrain model runways was 
subjectively evaluated by both Boeing and NASA personnel as very good. 
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Computer System 

Simulation of the flight vehicle aerodynamics, flight control and 
cockpit information display, and visual simulation system control were 
accomplished using a portion of the Boeing Visual Flight Simulator 
computer system. One XDS 930 digital computer was operated in 
conjunction with a Varian 622i digital computer, a Sanders ADDS 900 
graphics display system, and analog to digital and digital to analog 
conversion equipment. 

The mathematical model which described the dynamic flight of the H-33 
orbiter vehicle and the flight environment was programmed for real 
time solution on the main digital computer. The model was a relatively 
sophisticated description of the vehicle, including computation of 
the aerodynamic forces and moments, body axis/stability transformations, 
translation and rotational accelerations and velocities, and dynamic 
pressures as well as longitudinal landing gear dynamics, aerosurface 
and speedbrake dynamic pressures, stability augmentation system and 
flight control system operation. Included in the model were computa- 
tions for the flight environment in terms of wind accelerations, 
velocities, shear, and gusts. Figure 6 depicts the general charac- 
teristics of the H-33 vehicle. 

Input commands from the pilot in the cockpit and programmed environmen- 
tal conditions were used to compute the vehicle attitude, position and 
velocity information. This information was sent as operation commands 
to each axis of the visual simulator camera servo system, which 
oriented the high resolution TV cameras over the scaled terrain models. 
The resulting video signal was then processed and fed to the large 
high resolution TV display in the cockpit. The motions of the visual 
scene corresponded to what would be seen through the cockpit window. 
Simultaneously, vehicle attitude, position, and movement data was 
processed for display on the cockpit instruments. 
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Figure 6: H-33 Space Shuttle Orbiter General Characteristics 
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Throughout each simulation flight, the specified flight performance 
data was collected and stored. At the end of each flight, the 32 
flight performance measures were printed along with pilot's names, 
session and flight numbers, and corresponding experimental conditions. 
After each set of five flights, block summary data and standard devi- 
ations were printed. 

Procedure Task Function Logic System 

The procedure task function logic system (PTFLS) simulated subsystem 
failures and operated the procedure task displays and controls. 
Subsystem failures programmed on the PTFLS were initiated both 
manually and by computer control. As a function of pilot responses 
to the programmed failures, the PTFLS sequenced through the procedure 
logic, providing subsystem status information to the pilot on the 
cockpit displays. In response to pilot selection of primary or 
alternate stability augmentation system (SAS) elements, the PTFLS 
transmitted command signals to the computer system to fail or rein- 
state the SAS program for one or all vehicle axes. PTFLS command 
signals likewise directed computer programs to initiate auxiliary 
power and terminal navigation failures. Procedure task time and 
error data was recorded on the PTFLS FM magnetic tape for subsequent 
data analysis. 


Procedure 

The experimental procedure was broken down into three general phases. 
The first phase was the initial briefing and training of the pilot test 
subjects to perform the flight control and procedure task. This 
phase concluded with qualification testing of the pilots. The second 
phase was the retention interval. During this 4-month period of the 
subject's absence from the simulated space mission and normal flying, 
training performance data analysis was completed so as to permit 
assignment of the subjects to separate groups of comparable overall 
ability. The third phase involved applying the specific refresher 
procecure as established and assigned to each group and carrying out 
the retention testing of the subject population. 
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Initial Training 

The initial training involved an introduction to the problem, ground- 
school, and then the comprehensive training. Throughout training, each 
pilot had a complete mission description, flight control and procedure 
training handbook (Appendix). Each pilot received his training in a 
series of 1 hour and 2-1/2 hour training sessions over a 5-day period 
of time. Figure 7 depicts the training schedule flow that was used to 
bring the pilots to their qualification level of performance. 



Figure 7: Schematic of Pilot Training Schedule 

for Flight and Procedure Tasks 

Pilot training included groundschool on flight and procedure tasks 
(Figure 8), cockpit familiarization, procedure task training, visual 
flight and landing practice, full flight of instrument approaches to 
visual landings, and full mission flights including emergency proce- 
dure tasks. The training was continued until the means and standard 
deviations of selected performance parameters reached an asymptotic 
level cif acceptable flight performance. 
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Figure 8: Groundschool Briefing on Flight and Procedure Tasks 


The pi'ots were briefed that their individual scores would be based on 
their cumulative performance on the following: 

(a) Correct procedure taken to rectify emergency situations; 
the time taken to complete the emergency sequence. 

(b) Correct normal operating procedure. 

(c) Integrated errors determined from a nominal flight path 
between control check points or reference planes; and 
the instantaneous errors observed when crossing those 
control check points. 
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The most important of these performance parameters included measure- 
ment of lateral and vertical offset from the glide path, airspeed 
error, and rate of descent. Integrated and instantaneous measurements 
were taken when the vehicle crossed the vertical planes through the 
TACAN, at the nominal flare and threshold points, and at touchdown. 

The desired goal at touchdown was for a sink rate of from 4 to 6 feet 
per second, 2000 feet down the runway, on centerline, gear down, and 
yaw and bank angles near zero. Unsatisfactory performance at touch- 
down was defined by either of the following: sink rates greater than 

12 feet per second, touchdown short or wide of the runway, landing 
gear up : , and/or yaw or bank angles greater than 10 degrees. 

Although there were marked, noticeable skill differences between indi- 
vidual pilots, most subjects progressed through training uneventfully. 
Only one of the original training group of 10 pilots was terminated 
prior to qualification because of slow progress through the training 
program:; two others were dropped because of incompatible work duties 
during the retention interval. 

All pilots were trained and qualified on both flight control and emer- 
gency procedure tasks. The flight control tasks required an average 
of 48 flights per pilot to train to proficiency, with a range of 30 
to 76 flights. In terms of simulator training time, the pilots 
required an average of 6 hours at the controls to reach qualification. 
An average of 167 procedure task trials were required for each subject 
to reach qualification (range 100 to 250). The time expended for 
groundschool briefing and procedure task training averaged 2.8 hours 
per pilot. The average training time per pilot amounted to 8.8 hours 
over 5.3 sessions. 

Retention Interval: Method Assignment and Test Preparation 

Upon completion of training and collection of the training qualifica- 
tion performance test data, all training materials were recovered from 
the pilots. The pilots were informed that they were entering the 4 
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month retention interval phase. During the retention interval, they 
were not to return to the simulator laboratory, discuss the simulated 
flight, O"' perform any piloting functions in other flight simulators 
or actual aircraft. The pilots were told they would be contacted 
regarding their retention test schedule two weeks before the end of 
the retention interval. 

The study was designed to compare no practice retention with retention 
after various refresher techniques by testing three groups of approx- 
imately the same overall flight skill capabilities. During the course 
of the initial training and preliminary test set-up, it was apparent 
that the available subject population would contain noticeable individ- 
ual differences in basic flight skills. Also, because of budgetary 
and time restrictions the groups were limited to a small number of 
pilots per group (n = 5). 

Therefore, it was necessary to establish relative equality between the 
groups by judicious assignment of the subjects to groups by skill 
level. The technique used was similar to that used in a randomized 
blocks or matched groups design. The groups were matched with the 
expectation that the groups of subjects taken as a unit would be more 
homogeneous in their response to similar treatments than groups formed 
by selecting subjects completely at random. 

Based upon the scattergram plots of the flight control performance 
data obtained during qualification, the test subjects were ordered from 
most proficient to least proficient. The subjects were then assigned 
to treatment groups. In order to assess group equality, an analysis 
of variance was completed on each of 32 flight control performance 
measures using a subjects nested within groups design. A total of four 
of the AMOVAs showed significant between group differences. Within 
cell deviation scores were analyzed to determine which of the subjects 
accounted for these groups differences. Reassignment of 6 subjects 
was made between the groups and the data was prepared for a new series 
of ANOVAs. 
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The second series of 32 ANOVAs of the flight control performance data 
resulted in no significant group differences (p > .10). As would be 
expected,, significant between-subject differences as defined by the 
F for subjects within groups were found to occur for most of the per- 
formance measures. It was concluded that the revised group assignment 
of subjects provided the desired comparability of groups in performance 
across all flight control measures. With the assignment of pilots 
to groups based upon flight performance equality completed, an 
analysis of the 11 procedure performance parameters using the same 
experimental design was conducted. No significant differences between 
treatment groups were found for any of the procedural performance 
measures ,. 

The initial analyses of the flight test measures were based on data 
including the arithmetic sign. That is, the sign of the data indicated 
errors which were high, low, left, or right, short or long. As such, 
the data provided an indication of not only the magnitude of the error 
but also its relative direction. However, the arithmetic mean of such 
data does not reflect the true magnitude of the average error for the 
17 flight performance measures which involved direction of error. 
Therefore, another series of analyses were performed on the absolute 
error data (i.e., without regard to sign). Once again, no significant 
differences between groups (p > .10) were found for the 32 flight 
performance measures. 

The probability of significant differences between-groups and subjects 
nested within groups are depicted in Table 1 for each of the 43 perfor- 
mance measures obtained during the pilot training phase. In addition, 
the average performance for all subjects for each measure is listed. 

The mean performance values indicated that the subjects had achieved 
a high degree of performance proficiency at the end of training. 

The results of these analyses indicated that no significant group 
performance bias would affect the evaluation of retraining methods. 

With this assurance, the three groups were then randomly assigned to 
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Table 1: Pilot Performance and Analysis of Variance 

Results (F Ratio) at Completion of Training 



PERFORMANCE MEASURE 


ALTITUDE ERROR 
LATERAL ERROR 
HEADING ERROR 
VELOCITY 

DESCENT RATE ERROR 
INTEGRATED VELOCITY ERROR 
ALTITUDE ERROR 
LATERAL ERROR 
HEADING ERROR 
VELOCITY 

DESCENT RATE ERROR 
INTEGRATED VELOCITY ERROR 
INTEGRATED ALTITUDE ERROR 
INTEGRATED LATERAL ERROR 
ALTITUDE ERROR 
LATERAL ERROR 
HEADING ERROR 
VELOCITY 

DESCENT RATE ERROR 
INTEGRATED VELOCITY ERROR 
INTEGRATED ALTITUDE ERROR 
INTEGRATED LATERAL ERROR 
“LATERAL ERROR 
DOWN RANGE ERROR 
HEADING ERROR 
VELOCITY 

DESCENT RATE ERROR 
BANK ANGLE 
PITCH ANGLE 

INTEGRATED VELOCITY ERROR 
INTEGRATED ALTITUDE ERROR 
INTEGRATED LATERAL ERROR 
INITIAL RESPONSE TIME 
DECISION TIME 
SEQUENCE TIME 
TOTAL TIME 
DECISION ERRORS 
SEQUENCE ERRORS 
TOTAL ERRORS 

NUMBER PROCEDURES WITH ERROR 
NO LANDING GEAR 
AUX PWR RESPONSE TIME 
NAV FAIL RESPONSE TIME 


(FT) 

(FT) 

(DEG) 

(KTS) 

(FT/SEC) 

(KT-SEC) 

Iff) 

(FT) 

(DEG) 

(KTS) 

(FT/SEC) 

(KT-SEC) 

(FT/SEC) 

(FT/SEC) 

(FT) 

(FT) 

(DEG) 

(KTS) 

(FT/SEC) 

(KT-SEC) 

(FT/SEC) 

(FT/SEC) 

Iff) 

(FT) 

(DEG) 

(KTS) 

(FT/SEC) 

(DEG) 

(DEG) 

(KT-SEC) 

(FT/SEC) 

(FT/SEC) 

(SEC) 

(SEC) 

(SEC) 

(SEC) 


(SEC) 

(SEC) 



* p < 0.10 

** p < 0.05 

***p < 0.01 
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skill retention retraining groups with 5 subjects per group. These 
groups were: (a) no practice - checklists only; (b) static practice - 

training manual, photos, and checklists; (c) dynamic display - training 
manual, photos, checklists and recorded flights. Based upon retraining 
method assignments to the groups, a schedule was prepared which iden- 
tified the type of retraining each subject would receive, and the date 
he was die for skill retention testing. 

A critical aspect of experimental programs which suspend complex sim- 
ulator operations for periods of several months is the ability to main- 
tain a constant experimental test environment. Most critical to an 
experiment of this type is the ability to maintain the exact display/ 
control relationships, flight dynamics, and visual presentations at 
the end of the retention interval as was experienced by the pilots at 
the end of training. In anticipation of this problem, high fidelity 
calibration recordings of all flight control elements of the simulator 
were made during the subject training period. These calibration record- 
ings then provided the basis for recalibration prior to subject 
testing. 

One week prior to the date the first subject was to be retested, a com- 
plete checkout and recal ibration of all simulation equipment was 
accomplished. The dynamics of the two-axis handcontroller, rudder 
pedals and speedbrake controls were checked as well as the analog 
calibration of the electro-mechanical flight instruments. Scaling and 
sensitivity of the electronic flight instruments (EADI and MFD) were 
held constant by the digital computer program and hardwired circuit 
cards. 

As an additional measure, the cockpit flight control output voltages 
were recorded during the training phase and then reflown through the 
computer prior to testing to determine the empirical equivalence of 
the flight profile, display control operations, and the visual scene 
camera servo system. Subjective testing flights were flown by the 
experimenter pilots to provide a subjective evaluation of the 
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similarity of the aircraft's handling qualities and visual presentation. 
Comparisons of the flight performance data recorded during training and 
the data obtained prior to retention testing as well as the subjective 
evaluation indicated that the simulator was recalibrated to the con- 
dition that existed during the training and qualification testing. 

Refresher Practice and Retention Testing 

Prior to the return of the test subjects, all experimentors and lab- 
oratory personnel practiced all test operations to ensure that the 
experimental procedures were consistent with those previously used 
during training and were performed without error. To ensure consist- 
ency of testing across subjects within each skill retention group, a 
set of retention test procedures for the experimenter's procedures 
and written introductory instructions for each retention group were 
prepared. 

At the end of the 4 month retention interval, a different type of re- 
fresher training was provided to each of the three groups (Table 2). 
After each subject arrived in the simulation area, he received the 
written instructions which indicated to which skill retention training 
group he had been assigned, a brief description of the refresher 
training he would receive, the number of flights that he would make, 
and the amount of time required for testing. 

Method Group I . The No-Practice, Checklist-Only Group was tested at 
the end of the 4 month retention interval with only minimal re- 
introduction to the pilot task. Upon arrival in the simulator area, 
the pilot received and read the written introductory instructions and 
then was seated in the flight simulator cockpit. The seat and rudder 
pedals were adjusted and the pilot was allowed a few minutes to 
familiarize himself with the cockpit, the instruments, and control 
locations. During this time the pilot was allowed to review the 
flight and procedural checklist. 


23 



D1 80-1 5082-1 


Table 2: Method Group Refresher Training on First 

and Sixth Retention Test Flights 


GROUP I: GROUP II: GROUP III: 

NO PRACTICE STATIC PRACTICE DYNAMIC DISPLAY 


FIRST 

RETENTION 

FLIGHT 


SIXTH 

RETENTION 

FLIGHT 


• CHECK LISTS 
(DURING FLIGHT 
ONLY) 

• CHECK LISTS • CHECK LISTS 

• FLIGHT MANUAL • FLIGHT MANUAL 

• LARGE PHOTOS • LARGE PHOTOS 

• VIEW "CANNED" 
FLIGHT 

• WARMUP PRACTICE 

• CHECK LISTS 

• WARMUP PRACTICE • WARMUP 

• CHECK LISTS PRACTICE 

• FLIGHT MANUAL • CHECK LISTS 

• LARGE PHOTOS * FLIGHT MANUAL 

• LARGE PHOTOS 

• VIEW "CANNED" 
FLIGHT 


Review of this checklist provided the pilots with the operational 
flight plan, key altitude, velocity, and attitude information at 
specified control points (i.e. at the TACAN station, at flare, etc.) 
as well as procedural task operations. This checklist did not provide 
any indication of vehicle idiosyncrasies, descriptions of flight 
instrumentation, or detailed operational procedures. At the end of 
the review, the pilot was giver a last minute briefing on how to start 
and reset the simulator. 

The first flight was then started. The data from this flight provided 
the measures of retention performance without the benefits of practice 
(Table 2, Group I, First Retention Flight). After completion of the 
retention test flight, the pilot flew an additional four flights. Data 
was collected on all flights and at the end of each flight the only 
feedback information that the pilot received was the distance down the 
runway and descent rate at touchdown. 


During the additional four test flights, the pilot could become familiar 
with vehicle operation, instrumentation and visual cues received during 
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approach and landing. These four flights in combination with the 
retention interval test flight provided a total of five dynamic warmup 
practice flights. Upon completion of this series of five flights, the 
pilot was allowed a 10 minute break during which time he was permitted 
to get out of the simulator and stretch his legs. 

At the end of the rest period, each pilot in Group I was once again 
tested on his ability to successfully fly the simulated approach and 
landing mission. The data from this sixth retention test flight was 
used to assess the effects of dynamic warmup practice on skill 
retention (Table 2, Group I, Sixth Retention Flight). 

Method Group II . The Static Rehearsal Group received detailed brief- 
ings from the flight handbooks which covered flight vehicle character- 
istics, the displays and controls, flight operations, flight procedures, 
and emergency procedure operations at the end of the retention interval. 
Upon arrival in the simulation area, the pilot was provided with the 
written instructions which indicated to which group he had been assigned, 
the type of retraining practice he would receive and the number of 
flights he would fly. The pilot was then taken to the briefing room 
and provided with the flight control and procedural checklists, the 
training manual which was used during initial training, and a series of 
60 percent life-size photographs of the cockpit at key flight control 
points in the mission profile. 

These photographs depicted instrument information and the external 
visual scene as seen through the cockpit windscreen at each key control 
point. This static representation of the task environment at the 
critical control points, permitted the pilots to follow the essential 
elements of the flight through the descent, approach and landing. 

Through che use of these large scale photographs, the flight control 
information provided by the instruments was correlated with the visual 
scene. The pilot thereby gained an appreciation for vehicle altitude 
and attitude during the critical visual portion of the flight (Figure 
9). Each photograph was labeled with a brief description of the 
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Reproduced from 
best available copy. 


critical mission control point, and accompanied with the flight plan 
checklist, the pilot could project himself into the task environment. 



Figure 9: Static Rehearsal Group (II) Pilot Reviewing Large 

Scale, Critical Control Point Photographs 


I he pilot was permitted 40 minutes to review this material and prepare 
himself for flight testing. At the end of the 40 minutes, he returned 
to the simulation area and was seated in the simulator cockpit. The 
seat and rudder pedals were adjusted and a few minutes were allowed 
for re-"ami liarization of the cockpit using the checklists and the 
training manual. At the end of the familiarization period, the sub- 
jects were given a last-minute briefing on the operation of the 
simulator controls to start and stop the flights. The remaining 
sequence of events were identical to those for subjects in Group I. 
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The performance on the first retention test after the retention inter- 
val was compared to the qualification performance obtained at the end 
of training to the effects of static rehearsal training on skill re- 
tention performance. As with Group I, Group II also received an 
additional four test flights to complete the dynamic warmup practice 
(Table 2). The sixth retention test provided the basis for analyzing 
the benefits of combined static and dynamic refresher training. 

Method Group III . The Dynamic Display Group received the same refresh- 
er training provided to the Static Rehearsal Group. However, upon the 
completion of the briefing session and cockpit familiarization, tiie 
pilots were permitted to view three fully dynamic flights from the 
cockpit. These fully dynamic flights were produced from recorded con- 
trol signals from selected, previously flown flights. The recordings 
were played back through the computer simulation equipment to operate 
the cockpit displays and provide the out-the-wi ndow visual scene. 

During each of these flights, the pilot was instructed to pay partic- 
ular attention to the dynamics of the information presented by the 
cockpit instruments in relation to the visual scene depicted during 
the approach and landing phase of the mission. He was to observe 
the detailed dynamic progression of each flight, including minor pilot 
errors and corrective actions that could be used to successfully land 
the vehicle. Operating as an open-loop simulator, the Dynamic Display 
method provided an exact representation of the task environment 
throughout the entire mission. 

Dynamic display practice for the procedure task was similar to the 
flight control task with the exception that the pilots did not view 
an automated "canned" representation of the task operations. Instead, 
they watched an experimenter run through several operations of each 
procedire sequence. This approach provided procedure task rehearsal 
with dynamic sequencing of display information and control input 
responses which was comparable to the dynamic display method for flight 
control . 
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Once again, performance for this method group was measured during the 
first test flight. This data, compared to the performance obtained at 
the end of training, provided for the evaluation of dynamic display 
practice. Additional test flights were then flown to provide dynamic 
warmup for the evaluation of the combination of dynamic display and 
dynamic warmup practice (Table 2). 
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3. RESULTS 


Eleven performance measures were used to evaluate flight control per- 
formance. Six of the performance measures were repeated at four points 
in the flight, two measures were taken during three flight phases, and 
two additional measures were taken during the final phase. This pro- 
vided a total of 32 flight control data measurements. A total of 
eleven measures were used to evaluate performance for the procedure 
tasks. Performance was measured at the end of training (qualification 
test), at the end of the retention interval (retention test), and 
after five test flights (warmup test). 


Each of tnese performance measures were subjected to the analysis of 
variance statistic to evaluate the effects of no practice and refresh- 
er training on skill degradation. The two factor (retraining methods 
by performance tests) experimental design with repeated measures on 
the test factor (subjects nested within groups) is depicted in Figure 
10. It may be seen that the effects of methods are confounded with 
subject groups while the effects of tests and the test by method 
interactions are free of such confounding. However, there were only 
slight differences across method groups as the subjects were assigned 
to matched groups based upon qualification performance. 

PERFORMANCE TESTS 


GROUP I: NO PRACTICE 


GROUP II: STATIC 

REHEARSAL 


GROUP III: DYNAMIC 
DISPLAY 



ANOVA SUMMARY 


SOURCE 

DF 

BETWEEN SUBJECTS 

14 

METHODS (M) 

2 

Sc WITHIN GROUPS 

12 

WITHIN SUBJECTS 

30 

TESTS m 

2 

T x M 

4 

T x Sc WITHIN GPS 

24 

TOTAL 

44 


Figure 10: Experimental Design for Effects of No Practice and 

Retraining Practice Methods on Skill Retention 
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Flight Control Task 

A total of 32 measures of flight control performance were obtained and 
analyzed during the descent, approach and landing of the vehicle. 

Based upon three of these measures, one critical measure of operational 
significance was derived: landing success; that is, did the vehicle 

land (a) on the runway, and (b) with a descent rate within the tol- 
erance of the landing gear structural strength. In addition, 26 of 
the 32 ir dividual performance variables were integrated in a combined 
flight performance measure to assist in the overall interpretation of 
the results. 

Crash Larding Criteria 

The occurrence of a crash landing was used as one criterion of per- 
formance by which the retention interval practice methods were com- 
pared. A landing that was short of the runway or a touchdown so far 
down the runway (> 5,000 ft) as to cause the vehicle to run off the 
end, a landing that was off the runway to the right or to left, or a 
landing that was so hard as to collapse the landing gear (16 ft/sec) 
was a crash landing which could destroy the vehicle. Lateral error, 
down range error and descent rate at touchdown were evaluated as 
crash criterion measures for the three method groups. Summarization 
of this cata is depicted in Table 3. The frequency of occurrence of 
short (long), hard, and wide landings is presented as a function of 
each retention interval practice condition. 

It can be readily seen that the absence of any type of retention inter- 
val practice was disastrous. Each of the five pilots in Method Group 
I crash landed the vehicle at the end of the four month retention 
interval as defined by one or more of the crash condition criteria. 
Dynamic warmup practice afforded by the five practice flights reduced 
the number of crash landings to two. Static rehearsal practice 
(Group II) prior to testing at the end of the retention interval 
reduced the occurrence of crash landings experienced by the no practice 
group by three. This static rehearsal resulted in only two crash 
landings, which was the number experienced by the pilots with dynamic 
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warmup practice only. The addition of warmup practice to static 
rehearsal practice eliminated the incidence of crash landings complete- 
ly. Only the dynamic rehearsal for Group III resulted in no crash 
landings at the end of the retention interval. 


Table 3: Frequency of Crash Landings on Retention 

Test Flights as Function of Practice Method 


CRASH CONDITION 

GROUP 1 

GROUP II 

GROUP III 

NO 

PRACTICE 

DYNAMIC 

WARMUP 

STATIC 

REHEARSAL 

S-R AND 
WARMUP 

DYNAMIC 

DISPLAY 

D-D AND 
WARMUP 

SHORT 

(LONG) 

2 

1 

1 




HARD 

5 


2 



r 

WIDE 

2 

2 

1 


i 

1* 


TOTAL PILOTS 
WITH CRASH 

5 

2 

2 



1* 


•EXPERIMENTAL ARTIFACT, 
SEE TEXT 


The combination of dynamic rehearsal and warmup did produce one crash 
landing. In this case the pilot landed off the runway with a descent 
rate that exceeded the capacity of the landing gear. However, during 
the final landing phase of this flight, the visual scene produced by 
the simulator was disrupted a short period of time (approximately 5 
to 10 secs). Evaluation of the pilot's data and experimental test 
records showed that he was lined up with the runway at approximately 
the correct altitude when crossing the runway threshold. After the 
disruption, he ballooned the landing and deviated off course, followed 
by a stall and crash off the side of the runway. In this case, there 
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is good reason to believe that in the absence of any simulator mal- 
function the pilot would have landed successfully. Inspection of 
the immediately preceding and following flights confirmed his ability 
to land successfully. It was concluded that this crash was an 
experimental artifact and not considered further. 

The frequency of crash landings which occurred for each of the reten- 
tion interval practice methods was compared using the Chi square 
technique. A significant value of (p < .01; = 15.75 with 5 df.) 

was obtained, which indicated that the proportion of crash landings 
differed across the retention interval no-practice/practice method 
groups. The difference is a function of the high number of crash 
landings (5 out of 5) which occurred for the no practice group. 

Combined Flight Performance Measure (CFPM ) 

The CFPM is an expression of overall piloting performance throughout 
the entire flight in one measure. The measure was determined by 
equally weighting all of the error performance measures (except 
heading) at each of the four critical flight control points (TACAN, 
Flare, Threshold, Touchdown). Heading "errors" (deviations from an 
ideal course line) were not considered because these were generally 
less than 2 degrees and were usually indicative of a corrective action 
taken to cecrease the apparent lateral error at the moment. Likewise, 
pitch and bank angle error at touchdown were not included in the 
combined measure as the deviations were very small and usually 
corrective in nature. Thus, of the 32 flight performance measures, 

26 were used to derive the CFPM. 

A baseline performance level was determined for each measure by its 
average value in all qualification performance tests. This nominal 
or "qual" level was used to establish the performance factor or ratio 
for each data measurement that was taken. That is, the flight per- 
formance factor for a data measurement was the actual value measured, 
divided by the mean of that parameter in all qualification tests. 

Since the CFPM was evolved to give a picture of the flight overall, 
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all the parameters were given equal weight. The 26 flight performance 
factors were, therefore, arithmetically averaged to provide the overall 
combined flight performance measure for each flight. 

Overall flight control performance was evaluated using the CFPM for the 
total flight. Figure 11 depicts the effects of no practice and the re- 
training methods on skill retention for the total flight. The data is 
plotted relative to the average performance achieved by all subjects in 
the three method groups as indicated by the qua! level reference line. 



Figure 11: Skill Retention as a Function of Practice Method 

(Based on Combined Flight Performance Measure) 
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A tests; by methods ANOVA, with subjects nested within methods, was 
used to analyze differences between groups and performance tests on 
the combined flight performance measure. The retention test data 
evaluated the effects of no practice, static rehearsal and dynamic 
display rehearsal for Method Group I, II, and III, respectively. The 
warmup test data evaluated the effects of dynamic warmup practice, 
static rehearsal with dynamic warmup, and dynamic display rehearsal 
with dynamic warmup for the three groups. 

The results of this analysis are depicted in the overall flight part 
of Tab'e 4, with significant differences (p < .01) detected for both 
main effects and for the interaction. The significant methods effect 
indicated that retention performance improved as a function of practice 
method. The addition of warmup practice further improved retention 
performance for the three groups as shown by the significant tests 
effects. However, the methods by tests interaction showed that the 
benefit of warmup practice was most strongly associated with the groups 
that had less efficient or no retention training. The impact of warmup 
on Group III (dynamic display) performance was minimal since retention 
performance after 4 months was so high to begin with. 

The data were further analyzed using the Duncan's New Multiple Range 
Test. Performance of both the no practice group and the static 
rehearsal group was significantly degraded at the end of the retention 
interval (Duncan's, p < .05), while the dynamic display rehearsal 
group showed no significant and very little practical degradation. 
However, the static rehearsal group performance was significantly 
better than the no practice group (Duncan's, p < .05). Warmup 
practice significantly (Duncan's, p < .05) reduced the no practice 
degradation (Group I); however, performance still showed a very strong 
trend toward degradation when compared to qualification performance. 

The addition of dynamic warmup practice to static rehearsal likewise 
significantly reduced the amount of degradation for Group II (Duncan's, 
p < .05). In this case, however, the combination of the two methods 
reduced degradation considerably more than either method used singly. 
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and while some degradation still existed, it was probably not of 
practical significance. No significant differences were found between 
qualification performance and retention tests for Group III. 


Table 4: Analysis of Variance Results (F Ratio) for 

the Combined Flight Performance Measure 


MISSION PHASE 

SOURCE 

TESTS 

METHODS 

T x M 

OVERALL FLIGHT 

22.24*** 

4.58** 

5.59*** 

TACAN 

8.67*** 

1.45 

1.47 

FLARE 

4.61** 

20.00*** 

9.11*** 

THRESHOLD 

4.14** 

1.86 

1 

1.13 

TOUCHDOWN 

1.60 

4.38** 

1.23 


* P< .10 
** P< .05 
*** P < .01 


Similar results were obtained when performance during each flight 
phase was evaluated. The four flight phases were: 1) Start to TACAN 

(IFR); 2) TACAN to flare (VFR); 3) flare to threshold (VFR); and 4) 
threshold to touchdown (VFR). 

Figure 12 depicts performance as measured by the CFPM as a function of 
flight phase. The CFPM data for each flight phase were analyzed using 
the same ANOVA as for the overall flight; these results are also 
shown in Table 4. It can be seen that the dynamic display method was 
extremely powerful for the three phases which involved out the window 
vision (VFR). Only for the data taken at the TACAN (IFR) was the 
dynamic display method inadequate after 4 months without practice. A 
slight improvement over static rehearsal was noted, but the magnitude 
was of no practical or statistical significance. 
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TACAN FLARE 




Figure 12: Skill Retention as a Function of Practice Method for 

Four Critical Flight Phases (Based on Combined 
Flight Performance Measure) 
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Another difference noted in the data taken at the TACAN was the 
failure of static rehearsal plus dynamic warmup (Group II) to rein- 
state performance while dynamic warmup alone (Group I) did. This 
apparent inconsistency is easily understood in relation to the total 
piloting task during this flight phase. In addition to flying, the 
pilots were required to perform a number of emergency flight procedure 
tasks. The static rehearsal group (II) attempted to perform on both 
the flight control and procedure tasks. Without the benefit of rehear- 
sal, the pilots in the no practice group (I) had considerable trouble 
with the procedure task; as a result, most just ignored the procedure 
completely. Group I, therefore, spent all their time concentrating 
on flying while Group II worked on both tasks. This part task versus 
whole task difference accounted for Group 1 1 s dramatic improvement with 
warmup practice. As can be seen, the combined flight control per- 
formance measure showed the same overall results in terms of method 
selection as did the frequency of crash landings. 

Individual Flight Control Performance Measures 

In order to provide additional insight into the degradation process 
and effects of retraining methods, the results obtained from each of 
the 32 separate flight control performance measures are depicted in 
Table Ei. Based upon normal operational limits, the majority of the 
performance measures showed degradation of practical importance. How- 
ever, some measures can be considered to have little practical or 
operational significance for this task. For example, heading error 
at the threshold increased by a factor of 9 with no practice (Method 1, 
qual to no practice). However, the maximum error without practice was 
only 1.8 degrees, still considerably better than the normal perfor- 
mance limit of + 5.0 degrees for a VFR approach. 

It should be noted that for Table 5 an increase in numerical size 
indicates increased degradation for error measures of performance only. 
Deviation up or down from the qualification performance levels are 
indications of degradation for the velocity, bank angle and pitch 
angle measures. 
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The data were analyzed using the two factor, analysis of variance 
design with repeated measures on one factor (subjects nested within 
groups)- The results of the 32 analyses are depicted in Table 6. The 
data for each of the 32 individual measures showed trends comparable 
to the crash criteria and the combined flight performance measure 
data. However, due to the small sample size, the analysis failed to 
detect significant differences for many of the individual measures. 

Procedure Task 

Eleven measures of time and error performance were used to evaluate 
performance for the three procedure tasks: Subsystem Scan failure 

monitoring, Stability Augmentation System (SAS) failure monitoring and 
control and landing gear actuation. The data for each procedure per- 
formance measure were analyzed using the same two factor analysis of 
variance design with repeated measures on one factor as was used for 
the flight control analysis. Once again, the factors were retention 
methods and performance tests. 

As overall indicators of performance, the SAS procedure measures of 
total time and total errors were representative of all the procedure 
task data. They combine relatively pure psychomotor performance with 
the more complex decision/response performance. Analogous to the 
combined flight performance measure, these total performance measures 
are presented separately to provide an integrated view of the effects 
of retraining methods on procedure performance. 

SAS Total Time and Errors 

The results indicated that procedure performance degraded significantly 
after four months without practice. As overall measures of performance, 
the average total time required to perform the procedure task is 
depicted in Figure 13 and the average number of total errors per 
procedure is depicted in Figure 14 for each of the retention methods. 

No significant differences were found between the static rehearsal, 
dynamic display, and warmup retraining methods for the measure of 
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Table 6: Analysis of Variance Results (F Ratio) for 

Individual Flight Control Performance Measures 


PERFORMANCE MEASURE 

SOURCE 

TESTS 

METHODS 

TxM 


1 ALTITUDE ERROR 

5.79*** 

2.72 

0.46 


2 LATERAL ERROR 

2.94* 

0.09 

0.86 

z 

< 

3 HEADING ERROR 

2.81* 

1.60 

0.54 

a 

< 

4 VELOCITY 

2.22 

2.56 

3.01** 


5i DESCENT RATE ERROR 

5.40** 

1.39 

1.72 


(i INTEGRATED VELOCITY ERROR 

7.43** 

1.51 

4.12** 


7 ALTITUDE ERROR 

0.49 

2.02 

2.09 


{) LATERAL ERROR 

0.14 

1.32 

1.60 


E» HEADING ERROR 

0.20 

0.22 

0.69 

LU 

1C VELOCITY 

0.59 

0.09 

0.59 

cc 

< 

11 DESCENT RATE ERROR 

5.43** 

0.41 

2.51* 

_j 

LL 

i:; 1 INTEGRATED VELOCITY ERROR 

13.03*** 

0.66 

5.34*** 


13 INTEGRATED ALTITUDE ERROR 

4.35** 

1.24 

1.02 


1<!. INTEGRATED LATERAL ERROR 

4.51** 

2.86* 

0.37 


1!:i ALTITUDE ERROR 

0.34 

1.93 

2.52* 


1 1fi LATERAL ERROR 

0.35 

9.70*** 

3.07** 

o 

17 HEADING ERROR 

0.56 

0.75 

1.49 

I 

CO 

13 VELOCITY ERROR 

3.50** 

0.35 

0.47 

LU 

CC 

19 DESCENT RATE ERROR 

1.22 

3.02* 

2.96** 

X 

H 

20 INTEGRATED VELOCITY ERROR 

5.96*** 

0.23 

1.56 


21 INTEGRATED ALTITUDE ERROR 

0.04 

1.71 

2.78* 


22 INTEGRATED LATERAL ERROR 

0.59 

2.73 

2.01 


23 LATERAL ERROR 

1.21 


2.46* 


24 DOWN RANGE ERROR 

2.38 


0.57 


2!) HEADING ERROR 

2.33 


1.06 

~Z. 

5 

215 VELOCITY 

1.90 


0.24 

o 

Q 

27 DESCENT RATE ERROR 

6.46*** 


2.33* 

X 

o 

2) 

215 BANK ANGLE 

0.69 


1.53 

o 

I— 

29 PITCH ANGLE 

6.77*** 


2.29* 


30 INTEGRATED VELOCITY ERROR 

0.39 

1.22 

0.69 


31 INTEGRATED ALTITUDE ERROR 

2.01 

2.34* 

3.24** 


32 INTEGRATED LATERAL ERROR 

0.39 

0.76 

0.65 


* p < o.io 
**p < 0.05 

*“p <0.01 
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total time. The highly significant tests effect (p < .005) was due 
primarily to the large amount of degradation experienced by Group I 
with no practice at the retention test. This same condition was 
probably the primary factor in the strong methods effect trend. How- 
ever, the 100 percent degradation experienced for the static rehearsal 
method after 4 months without practice must be considered a contributor 
to the methods effect trend. 



TRAINING 


INACTIVITY 
(4 MONTHS) 


ADDITIONAL PRACTICE 
(25 OPERATIONS) 


QUALIFICATION RETENTION 

TEST 


WARMUP 

TEST 


Figure 13: Average Total SAS Procedure Time as a Function of Practice Method 
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The results were similar for the measure of total errors. The only 
difference was that static rehearsal without warmup showed no practical 
or significant trend toward degradation. While the dynamic display 
group showed a greater total error average, this was a general eleva- 
tion across tests. Retention performance in relation to performance 
at the end of training showed that the magnitude of degradation of 
the dynamic display group was comparable to that of the static 
rehearsal group. 



Figure 14: Average Number of Errors Per SAS Procedure 

Operation as a Function of Practice Method 
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Individual Procedure Performance Measures 

The effects of the practice methods on all of the procedure performance 
measures are depicted in Table 7. In addition to SAS total time and 
error, measures of the discrete elements of the SAS failure procedure, 
the response times associated with the Subsystems Scan (Auxiliary Power 
and Terminal Navigation failures), and landing gear status at touch- 
down are included. 

Table 7: Mean Performance as a Function of Practice 

Methods for Procedure Performance Measures 



PERFORMANCE 

GROUP 1: NO PRACTICE 

GROUP 11: STATIC REHEARSAL 

GROUP III: DYNAMIC DISPLAY 


MEASURE 

QUAL 

RET TEST 

WARMUP 

QUAL 

RET TEST 

WARMUP 

QUAL 

RET TEST 

WARMUP 

33 

INITIAL RESPONSE TIME 

1.76 

6.73 

2.44 

1.99 

3.00 

2.07 

1.84 

1.99 

1.94 

34 

DECISION TIME 

.99 

6.91 

1.23 

1.21 

2.56 

1.31 

1.14 

1.47 

1.23 

35 

SEQUENCi: TIME 

.56 

6.99 

.84 

.60 

1.27 

.73 

.66 

.75 

.59 

36 

TOTAL TIME 

5.07 

32.97 

8.31 

5.56 

11.45 

6.39 

6.48 

7.52 

6.59 

37 

DECISION ERRORS 

.2 

5.6 

2.2 

0 

.6 

.4 

1.0 

1.6 

.8 

38 

SEQUENCE ERRORS 

.4 

3.0 

1.4 

0 

0 

0 

.2 

.6 

.8 

39 

TOTAL EF RORS 

,6 | 

8.6 

3.6 

0 

.6 

.4 

1.2 

2.2 

1.6 

40 

NUMBER PROCEDURES WITH 
ERRORS 

.2 ! 

' 3.8 

2.2 

0 

.6 

1 .2 

.4 

1.0 

-6 

41 

NO LANDING GEAR 

0 

2 

0 

0 

1 

0 

0 

0 

0 

42 

AUXPWR RESPONSE TIME 

2.78 

1 11.30 

5.04 

2.52 

1 4.18 

1 3.04 

i 2.54 

2.86 

2.56 

43 

NAV FAIL RESPONSE TIME 

2.88 

82.56 

29.78 ! 

4.14 

3.50 

3.20 

2.70 

2.54 

2.42 


The results of the statistical analyses for each measure of procedural 
performance are depicted in Table 8. As can be seen, the data and the 
results of the statistical analyses for these measures were similar to 
the SAS Failure total time and error measures. Performance without 
practice degraded significantly and the application of dynamic warmup 
practice reduced the degradation to performance levels which approached 
those achieved at the end of training (Group I). 


While not statistically significant, there was a definite trend toward 
degradation for the static rehearsal method (Group II). For all 
practical purposes, the addition of dynamic warmup practice to static 
rehearsal reinstated performance to training qualification levels. 
After 4 months without practice, the dynamic display retraining method 
(Group III) virtually maintained performance at the qualification 
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levels. The extremely minor performance differences were generally 
further reduced by the addition of warmup practice; however, the 
differences were so small that not even a hint of statistical trends 
were detected. 


Table 8: Analysis of Variance Results (F Ratio) for 
Procedure Performance Measures 


PERFORMANCE MEASURE 

SOURCE 

TESTS 

METHODS 

Tx M 

33 

INITIAL RESPONSE TIME 

11.57*** 

6.60** 

6.16*** 

34 

DECISION TIME 

15.22*** 

5.58** 

7.19*** 

35 

SEQUENCE TIME 

4.84** 

3.31* 

3.47** 

36 

TOTAL TIME 

8.02*** 

3.06* 

4.08** 

37 

DECISION ERRORS 

10.39*** 

6.06** 

5.46*** 

38 

SEQUENCE ERRORS 

2.15 

6.18** 

1.52 

39 

TOTAL ERRORS 

8.56*** 

6.45** 

4.88*** 

40 

NUMBER PROCEDURES WITH ERRORS 

11.37*** 

1 6.55** 

4.50*** 

41 

NO LANDING GEAR 

3.60** 

1.20 

1.20 

42 

AUX PWR RESPONSE TIME 

14.96*** 

1 9.17*** 

7.88*** 

43 

NAV FAIL RESPONSE TIME 

3.90** 

4.79** 

3.99** 


* P< .10 

** P< .05 

*** p < .01 
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4. DISCUSSION 

During training, many of the pilots made unsolicited comments relative 
to the simulation and the tasks they were required to perform. The 
simulation was well accepted by the pilots with a consensus that the 
simulation was very good and required maximum concentration of pilot 
abilities throughout each flight phase. The general impression 
obtained, from observation throughout the course of the study, was 
that the pilots were faced with a difficult task. Both observation 
and pilots comments indicated that the emergency procedure task during 
flight tended to produce a task load situation comparable to that 
found in actual flight emergencies. The task became particularly 
stressful when the pilot made errors while performing emergency 
procedures. 

The general conclusion was that the simulation was a realistic reproduc- 
tion of c. representative approach and landing for a large flight vehicle. 
The test subjects were all experienced pilots. While their experience 
with the simulated vehicle was less than normally found in operational 
situations, they were well trained to a consistent and asymptotic level 
of operationally acceptable performance. The greater number of perfor- 
mance measures, their combination into a total measure (the CFPM) , the 
avail abi lity of an operationally practical measure of performance 
(crash landings), and the use of pilots as subjects resulted in a more 
stable ard consistent assessment of flight control skill degradation 
than in a previous study of instrument flight control of a booster 
launch (Sitterley and Berge, 1972). 

The results showed that pilot performance after 4 months without practice 
was significantly degraded for both flight control and procedural tasks. 
The magnitude of visual flight control degradation was comparable to 
that found in the previous study of booster flight control. In the 
present study, the instrument flight control phase also resulted in 
almost twice the amount of degradation experienced during the visual 
portion of the flight. 
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It has been postulated that visual flight control (VFR) would be sub- 
ject to greater skill degradation than instrument flight control ( I FR ) 
because of the greater complexity of the perceptual/motor coordination 
and integration process. The finding that this was apparently not the 
case (Figure 12) may be the result of relatively higher task loading 
of the IFR portion of the flight. In this study, the pilots were 
required to both fly and respond to procedure emergencies. Even though 
the emergency procedures were seldom successful without adequate 
practice, their occurrence, and the pilots' inability to nandle them, 
had a disturbing and disrupting influence on flight control performance. 

The procecure task results showed greater magnitudes of degradation 
than that found for flight control. However, the magnitude of degra- 
dation on the SAS procedure was less than found by Sitterley and 
Berge (1972). While the same basic procedure logic was used for both 
the SAS procedure and the procedure previously reported, the test 
subjects in this study were considerably more experienced in handling 
task loaded, multiple operations. 

While reviews of many previous studies have indicated that dynamic 
warmup practice is required to maintain continuous flight control 
skills (Naylor and Briggs, 1961; Gardlin and Sitterley, 1972), this 
study failed to find a difference in performance between carefully 
structured static rehearsal and dynamic practice. Further, neither 
the static rehearsal nor the dynamic warmup practice in the amounts 
provided were sufficient to reinstate performance to the levels 
achieved at the end of training. 

It is postulated that the dynamic warmup practice reinforced the 
psychomotor and perceptual elements of the task. However, without 
the thorough review of the operational flight plan and recommended 
flight procedures afforded by the static rehearsal, the amount of 
dynamic practice would have to be substantially increased to rein- 
state the coordination and timing of maneuvers and operational 
subtleties of the task. On the other hand, static rehearsal rein- 
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forced the understanding of operational procedures, appropriate control/ 
decision responses, and recommended maneuver sequences. Further, it is 
probable that to some extent, the perceptual flight cues were partially 
reinforced through the use of the series of photographs. However, the 
static rehearsal method did not adequately handle the perceptual/ 
vehicle response coordination cues. 

This interpretation of the effects of the dynamic warmup and the static 
rehearsal methods was supported by the data obtained from Group II. 

When the two techniques were combined, no crash landings were observed 
and degradation as measured by the CFPM was virtually eliminated. 
Apparently the two methods were highly complementary, each providing 
the skill reinforcement the other lacked. 

In some situations, static rehearsal alone might be considered for 
retraining. The results of this study showed that several aspects of 
flight control and procedural performance may be maintained using 
static rehearsal. On the other hand, while experimentally important 
in the evaluation of retraining methods, dynamic practice would not 
be used by itself in a real world training situation. There is no 
reason of practicality or feasibility not to include static rehearsal 
if a dynamic practice retraining is used. The combination of both 
methods, with the amount of dynamic practice increased over the levels 
used in this study, can be expected to reinstate all aspects of flight 
and procedural performance. 

One of the most interesting results was that no practical or signifi- 
cant degradation was found for the dynamic display rehearsal group. 

They received the same refresher training as did the static rehearsal 
group with one important difference. That difference was the inclusion 
of a more complete representation of the visual flight environment 
which provided the appropriate dynamic perceptual cues (e.g. depth, 
closure rate, and parallax). The pilots did not, however, have the 
benefits of control response feedback. The dynamic representation was 
preprogrammed and as such can be considered as an open-loop simulator/ 
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trainer method. As was noted from the separate analyses of the flight 
phases, the benefits of this method were most strongly apparent for 
the highly visual (VFR) portions of the flight. Relatively small 
reductions in degradation over static rehearsal were found for the 
instrument flight phase. Further, no crash landings were observed 
to occur on the first retention test for the dynamic rehearsal method 
group. Apparently, for this practical measure of performance, the 
combination of the static rehearsal package and the dynamic open-loop 
visual presentations were sifficient to counter degradation of those 
skills dependent upon the perceptual/vehicle response coordination 
cues. 

It might be postulated, then, that the integration or coordination of 
far field perceptual cues, which were so well reinforced by the 
dynamic rehearsal method, was the critical element of the retraining. 
Why the dynamic display rehearsal method, without control response 
feedback, was consistently better than dynamic warmup practice, or even 
the combination of dynamic warmup and static rehearsal, is not clear. 
The answer may be that the pilots were able to more fully concentrate 
on the perceptual cues in relation to the vehicle flight and instru- 
ments during dynamic display rehearsal than when they shared the per- 
ceptual cue retraining time with the task of flying during dynamic 
warmup practice. 

When the reinforcement of the perceptual cues was not as important 
(IFR portion measured at TACAN, Figure 12) or as efficiently accom- 
plished, the addition of dynamic warmup practice eliminated degradation 
The retraining methods which did not include display/ control response 
integration were insufficient. This finding was as might be expected 
since no far field perceptual cues were involved and the primary task 
depended upon careful control response coordination. 

The overall indication that dynamic rehearsal was better than warmup 
and that the addition of warmup to it produced negligible further 
improvement, suggested that the primary skill retention problem was 
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related to the maintenance of the visual/perceptual elements of the 
flight control skills. Certainly manual control performance did de- 
grade; however, with highly experienced pilots, the basic skill of 
integration of discrete control elements into a smooth, coordinated 
response, was more resistant to degradation. As such, it would appear 
appropriate to concentrate on enhancing the reinforcement of the 
understanding of the mission profile and flight operations in relation 
to the visual environment and perceptual cues. 

Tht retraining method with the greatest enhancement potential and cost 
benefit is static rehearsal. From the standpoint of practical signif- 
icance and the probability of a safe landing, it can be concluded that 
the dynamic rehearsal method was very successful and was equalled only 
by the combination of both the static rehearsal and dynamic warmup 
methods. Certainly some dynamic rehearsal method improvement potential 
existed, but it was associated with control response feedback afforded 
by the inclusion of dynamic practice. On the other hand, static 
rehearsal was shown to be as effective as limited dynamic warmup 
alone. While the present static rehearsal method failed to eliminate 
skill cegradation, it was, in fact, partially successful. 

One of the major failings of the static rehearsal method apparently 
was related to the inadequate presentation of the visual environment 
and critical perceptual cues. In studies of training device fidelity, 
Grimsley (1969a and 1969b) compared operational hardware with artist's 
reproductions which carefully represented the critical operational 
cues. For missile system procedural tasks, he found that the low 
fidelity equipment was equally effective as the high fidelity equip- 
ment ir terms of amount of skill retained and time to retrain. 

In both Grimsley's and the present problem, the integration of a 
critical sequence of events with the perceptual process produced the 
desired manual response. The timing, coordination, and control feed- 
back are definitely more an integral part of a flight control task, 
but the results obtained from the dynamic rehearsal method do indicate 
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potentially significant improvements are possible in the static 
rehearsal method. 

Improvements in the static rehearsal can be made in three principal 
areas: First, more information along the flight path should be 

included. In this study, only eight points in the flight were depicted, 
three of which were under IFR conditions not requiring perceptual cue 
integration. More representations of altitude and line up before 
reaching the flare point were needed Second, off-nominal pictorial 
representations should be included to permit comparisons to the normal 
flight profile reference, thereby giving the pilots a basis for recog- 
nizing poor performance. This approach was used for the dynamic 
rehearsal method and was subjectively important. Third, more active 
involvement with the static pictorial information is required in 
order to strongly establish the critical perceptual cues in the visual 
environment. This involvement should include: (a) prediction of 

future - :r light path from both nominal and off-nominal positions based 
upon visual and instrument indications of present attitude, position, 
and rates; (b) development, and correlation of instrument and pic- 
torial information; and (c) definition of required corrective actions 
and procedural operations in appropriate time sequence. 

Conclusions 


The conclusions derived from the results of this study may be summa- 
rized as follows: 

1. After 4 months without practice, experienced pilots encountered 
significant skill degradation on simulated operational flight 
tasks involving the use of far-field (out-the-wi ndow) visual 
cues . 

2. Procedure task skills degraded significantly after 4 months. 

The degradation was less than previously found for highly 
practiced non-pilots, even with conditions of higher task 
loading. Either the dynamic display or static rehearsal prac- 
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t'i ce was sufficient to reinstate procedure task performance. 

3. As expected, dynamic closed-loop warmup practice, in conjunction 
with a carefully structured static rehearsal briefing of the 
vehicle and cockpit characteristics and the mission profile was 
highly effective in reinstating flight skills. While either 
static rehearsal or dynamic warmup significantly reduced the 
magnitude of no-practice degradation, neither practice method 
was able to reinstate performance by itself. Apparently each 
method provided skill reinforcement the other lacked. 

4. Dynamic display practice, without the benefits of closed-loop 
warmup, totally prevented skill degradation for all portions of 
visual flight control. This method graphically demonstrated the 
substantive requirement of critical visual cue and flight opera- 
tion reinforcement for skill retention training. The success 

of this method, and the partial success of the static rehearsal 
method, suggested that alternate methods of retraining which do 
not involve dynamic interaction may be feasible. 
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6. APPENDIX - SUBJECT TRAINING PACKAGE 
Subject Training and Retention Test Briefing 

This appendix contains the materials initially presented to the test 
subjects at the beginning of their training and later used for re- 
hearsal briefing. During the pretraining orientation the material 
was supplemented with verbal clarification during the groundschool 
and cockpit familiarization sessions. The pilots were permitted to 
refer to or study the training materials during the course of training. 
After completion of the training performance qualification test, no 
access to any of the training materials was permitted. 

Prior to the retention test, written instructions were provided to the 
test subjects. These instructions also provided the introduction to 
the refresher training for Group II and III. These two groups used 
the same Flight Control and Procedure Training Package that was used 
during initial training. The retention test instructions were as 
follows: 

Group I. No Practice - Checklist Only 

Good Morning: During this next session you will complete your retention 

study flight testing. You have been assigned to the No Practice Group. 
That is, you will receive no preliminary briefings or refresher training 
prior to your retention flight testing. You will be seated in the 
simulator cockpit and the test flights will begin. At that time you 
will have access to two sets of checkl ists— (1 ) an approach procedures 
outlines; and (2) a SAS and subsystem failure procedures summary. 

No questions will be answered or assistance given concerning flight 
procedures or techniques until completion of your flights. You will 
fly ten approaches to landing with the H-33 orbiter simulator from the 
initial approach fix, starting at altitude of 31,400 feet. These 
flights are similar to your qualification flights and will contain 
failures in the navagation instruments, displays, and flight systems 
that you have previously been trained to handle. 
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We realize that your ability to fly a precise mission has probably de- 
graded. That is what we will be measuring. However, do the best you 
can. Remember that you are flying a return flight to an earth landing 
after a prolonged layoff. A safe, smooth flight with positive re- 
sponses is desired. 

This final test session will last approximately 2 hours. Are you ready? 
Group II. Static Practice 

Good Morning: During this next session you will complete your reten- 
tion study flight testing. You have been assigned to the Static 
Practice Group. That is, you will have 40 minutes to study: (1) your 

training manual, which details the mission and tasks required; (2) a 
series of 8 large-scale photos showing the cockpit instruments, displays, 
and out-the-window views at critical points within the approach to 
landing; and (3) two checkl ists--one, an approach procedures outline, 
and the other, a SAS and subsystems failure procedures summary. Once 
you are seated in the flight simulator, you will have a few minutes to 
become reacquainted with the cockpit first-hand. During the cockpit 
review you will have access to the training manual and your checklists; 
during flight you will have only the checklists. 

No questions will be answered or assistance given concerning flight 
procedures or techniques until completion of your flights. You will 
fly ten approaches to landing with the H- 33 orbiter simulator from 
the initial approach fix starting at an altitude of 31,400 feet. 

These flights are similar to your qualification flights and will contain 
failures in the navigation instruments, displays, and flight systems 
that you nave previously been trained to handle. 

We realize that your ability to fly a precise mission has probably de- 
graded. That is what we will be measuring. However, do the best you 
can. Remember that you are flying a return flight to an earth landing 
after a prolonged layoff. A safe, smooth flight with positive re- 
sponses is desired. 
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This final test session will take approximately 2-1/2 hours including 
briefings and flights. 

We will row proceed with the refresher training. 

Group III. Dynamic Display 

Good Morning: During this next session you will complete your reten- 

tion stucy flight testing. You have been assigned to the Dynamic 
Display Group. That is, you will view a representative sample of 
three approach flights to touchdown, after reviewing your written 
training material. In the review, you will have 40 minutes to study: 

(1) your training manual, which details the missions and tasks required; 

(2) a series of 8 large-scale photos showing the cockpit instruments, 
displays., and out-the-wi ndow views at critical points within the 
approach to landing; and (3) two checkl ists--one, an approach pro- 
cedures cutline, and the other a SAS and subsystems failure procedures 
summary. Once you are seated in the flight simulator, you will have 
five minutes to become reacquainted with the cockpit first-hand. During 
the cockpit review you will have access to the training manual and 
your checklists; during flight, you will have only the checklists. 

No questions will be answered or assistance given concerning flight 
procedures or techniques until completion of your flights. You will 
fly ten approaches to landing with the H-33 orbiter simulator from 
the initial approach fix starting at an altitude of 31,400 feet. 

These flights are similar to your qualification flights and will contain 
failures in the navigation instruments, displays, and flight systems 
that you have previously been trained to handle. 

We realize that your ability to fly a precise mission has probably de- 
graded. That is what we will be measuring. However, do the best you 
can. Remember that you are flying a return flight to an earth landing 
after a prolonged layoff. A safe, smooth flight with positive responses 
is desired. 
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This final test session will take approximately 2-3/4 hours including 
the briefings, retraining, and test flight. 

We will now proceed with the refresher training. 
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INVESTIGATION OF DEGRADATION OF LEARNED SKILLS 
TRAINING PACKAGE 


I. INTRODUCTION 

The objective of this study is to measure the degradation in perfor- 
mance observed in trained pilots returning after a prolonged lay-off. You 
have been asked to participate in this experiment because of your previous 
flight qualifications and your expectations not to participate in the control 
of an aircraft or other flight simulation for the next four months. 

Your part in the study will involve: (1) study of the simulator char- 

acteristics and flight profile, (2) completion of a ground school, (3) simu- 
lator flight training, (4) a test period, (5) a 4 month period of "no 
flying", (6) some phase of a refresher training program, and (7) a retest. 

This study has been designed to test your flight skills and performance 
during the approach and landing phase of a landable space vehicle. The 
flights will consist of normal, degraded, and emergency system operations. 

Your performance will be measured by our systems computer and will be judged 
in relation to an optimally designed flight profile. 

You will be expected to participate in approximately four half-day 
sessions during the space of a week for the initial participation and for one 
period during your retest. 

Our time schedule and simulator budget are limited. You are expected to 
train up to a nominal flight profile as quickly as possible and to fly all 
missions to the best of your abilities. Your instructor will be happy to 
answer any questions consistent with the training and our schedule limitations. 
Your participation in these tests is appreciated. 
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II. SIMULATOR/ VEHICLE CHARACTERISTICS 

The subject will "fly" simulator cockpit displays controlled by flignt 
cnaracter sties approximating those of the H-33 orbiter vehicle (Figure 2-1). 
The cockp't is configured with a spartan instrument panel and a simple set 
of controls. The vehicle attitude is controlled through an electric stick 
side-arm controller operating through a Stability Augmentation System (SAS). 
Instrumentation provides flight attitude, altitude, airspeed, and information 
as to vehicle position relative to radio navigation aids. An out-the-window 
VFR display provides a reference to touchdown on a TV terrain model of Ed- 
wards Air Force Base. 

The vehicle is unpowered during its reentry and approach. A safe flight 
requires the judicious utilization of energy management principles and adher- 
ence to the designed flight profile. 

Speed boards are provided to control the rate of descent as a function 
of airspeed. 

Optimum vehicle handling characteristics are obtained at an Indicated 
Airspeed (IAS) of 240 knots. Angle of descent, clean, is approximately 7 
degrees. At 60% speed boards and 240 knots, angle of descent is 10°. A 
speed of 210 knots will stretch your clean glide by 20%. 

III. SIMULATOR COCKPIT 

A. GENERAL 

The following paragraphs describe the cockpit configuration and opera- 
tion of the displays and controls necessary to fly the simulated flight pro- 
file of the H-33 orbiter from 30,000 feet to landing. 

The one-man cockpit is configured to provide all displays and controls 
necessary to fly the simulated mission. Display information is provided by 
both electromechanical and cathode ray tube displays. External VFR views are 
simulated using a virtual image system. This system provides a field of 
view of approximately 40 degrees through the centrally located windscreen. 
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The pilot's seat and rudder pedals are adjustable to permit the individual 
pilot to select a comfortable position consistent with the positioning of 
himself at the design eye reference point. The seat adjustments include 
height, fore and aft movement, back, and armrest position adjustment. The 
rudder pedals may be adjusted either fore or aft to compensate for seat 
position and leg length. 

Figure 3-1 depicts the cockpit display/control layout. Directly in 
front of the pilot, located on the center panel, is the Electronic Attitude 
Director Indicator (EADI). The CRT display provides basic attitude and com- 
mand information to the pilot. Directly below the EADI is the Mul ti -Function 
Display (MFD). This is a CRT display which provides a pictorial representa- 
tion of the horizontal situation and ILS heading and glideslope data. To 
the left of the CRT displays are located an electromechanical Calibrated 
Airspeed Indicator and standard Radio Direction Indicator. 

To the right of the CRT displays are located an electromechanical radar 
altitude indicator and rate of climb indicator. A master failure warning 
indicator is installed between these two instruments. When illuminated, this 
failure warning indicator directs the pilot's attention to either the 
Stability Augmentation Subsystem Selector Panel located on the upper left 
instrument panel or to the Subsystem Monitoring Panel located on the upper 
right instrument panel. 

On the side panel to the left are located landing gear controls and indicators, 
as well as the simulation operation controls which cannot be seen in the 
figure. On the side panel to the right is the Speedbrake Indicator. Below 
the Speedbrake Indicator is the instrument panel light control and rudder- 
pedal positioning handle. The throttle and speedbrake controls are located 
in the front of the instrument panel to the left, and a two-axis pitch and 
roll sidearm controller is located in front of the instrument panel to the 
right. Included in the sidearm controller is the pitch trim control. A 
detailed description of each of the displays and controls is provided in 
the following paragraphs. 
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Figure 3- 1: COCKPIT DISPLA Y /CONTROL CONFIG UR A TION LA YOUT 
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B. INSTRUMENTS/DISPLAYS 
Electron ic Attitude Pi rector Indicator ( EADI ) 

The EADI is the primary source of attitude information and flight path 
command information. The EADI display format is depicted in Figure 3-2. At 
the top of the display on either side of the vertical white bar are three 
reference markers which indicate roll attitude in ten degree increments. Di- 
rectly below the roll reference markers is a white aircraft symbol. The air- 
craft symbol and the roll reference markers remain in a fixed position on the 
display and are always superimposed upon other display data. Rotation of the 
horizon line and a perpendicular roll index bar provides roll attitude infor- 
mal on. 

Pitch reference markers are located below and above the horizon line 
in 10° increments. The pitch reference markers remain parallel to the 
horizon line regardless of rotational angle and are depicted as white lines 
against a shaded background below the horizon and as black lines against a 
light background above the horizon line. Pitch attitude information is 
depicted by a vertical movement of the horizon line and associated pitch 
reference markers in relation to the aircraft symbol. The EADI depicts 
the vehicle's flight path angle as a white line parallel to the horizon 
line and pitch reference markers. As the flight path angle changes, the 
flight path angle reference line (gamma) bar moves above or below the horizon 
line. 


ILS localizer and glideslope command bars are depicted in black on the 
display. The localizer command bar is oriented perpendicul ar to the air- 
craft symbol; it moves laterally to the left or to the right. When a local- 
izer bar is depicted to the left of the center of the airplane symbol, it 
indicates that the airplane is flying to the right of the desired heading 
and must be turned left to intercept the bar. The glideslope indicator bar 
is oriented parallel to the aircraft symbol and moves vertically. When the 
glideslope bar is depicted below the airplane symbol, it indicates the air- 
plane is "lying above the desired glideslope and the aircraft must be 
pitched down to intercept the glideslope. 
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10° PITCH REFERENCE MARKER 
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Figure 3-2: ELECTRONIC A TTITUDE DIRECTOR INDICA TOR (EADI) DISPLA Y FORMA T 
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In addition to the attitude and command information, the EEADI provides 
altitude information in the upper right hand corner of the display. The 
three digit altitude information is displayed in hundreds of feet in order to 
provide rapid recognition during the early part of the approach when altitude 
varies rapidly. 

Multi-Function Display (MFD) 

During the approach phase of the mission, the MFD provides a digital 
readout of several flight instrument parameters, and a graphical horizontal 
situation display representing the aircraft in relation to the TACAN station 
and the runway. The display format of the MFD during the initial portion of 
the approach phase is depicted in Figure 3-3. Mach number in hundredths, 
equivalent airspeed to a tenth of a knot, and altitude in feet are depicted 
at the top of the MFD. The primary aircraft heading indicator is located 
directly below these digital readouts. Heading is represented dynamically by 
a ribbon meter with a fixed index pointer. This meter shows current aircraft 
heading on a two-digit degree scale which moves either to the right or to 
the left. When the aircraft is turned to the right, the meter scale will 
move to the left such that the pilot "flys to" the desired heading. Directly 
below the heading indicator is the TACAN course indicator depicting the 
heading to the TACAN station and the distance from the TACAN station. 

At the bottom of the MFD is the graphical horizontal situation display. 
This display depicts a circle with a radius of 15 miles around the TACAN sta- 
tion. The TACAN is represented by a cross in the center of the circle. 

There is a line from the TACAN station down to the runway shown at the top 
of the circle. The horizontal position of the aircraft is dynamically 
represented with a cross. Extending from the aircraft symbol are four dots 
predicting the aircraft's future position. Each dot represents 10 seconds 
for a total 40 second prediction of the aircraft's location. 

As the aircraft passes over the TACAN station, the course indicator 
will change from "To TACAN" to "From TACAN." At the same time the digital 
velocity and altitude information will be replaced by a graphical represen- 
tation of the aircraft flying down the glideslope as depicted in Figure 3-4. 
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Figure 3-3: MUL 77 FUNCTION DISPLA Y (MFD) INITIAL APPROACH PHASE DISPLA Y FORMA T 











68 








D1 80- 15082-1 


Once again, the aircraft position on the glideslope will be indicated by a 
cross with four 10-second predictor dots. A short vertical line immediately 
to the left of where the glideslope intersects the runway threshold indicates 
the nominal flare point at 750 feet of altitude. 

Calibrated Airspeed (CAS) Indicator 

The CAS instrument is calibrated to read airspeed from approximately 
280 knots to 70 knots. The CAS is the primary velocity indicator for the 
approach and landing phase. 

Radio Direction Indicator (RDI) 

The RDI presents aircraft magnetic heading plus ILS and DUE information 
A digital course line can be manually selected by the pilot. As the aircraft 
turns, the compass card rotates. The glideslope and localizer bars indicate 
the centerlines of the glideslope and course line relative to the center of 
the instrument; thus, corrections are made by flying to an indicated offset. 

Radar Altitude Indicator 


The radar altimeter will not read or move dynamically above an altitude 
of 2,200 feet and is to be used only in the final landing phase of the flight 
The altimeter pointer reads hundreds of feet to a resolution of 200 feet per 
increment with thousands of feet indicated digitally in the window on the 
altimeter face. 

Rate of Climb Indicator (ROC) 

The ROC is a standard electromechanical instrument displaying vertical 
velocities to a maximum of 2,000 feet per second. As the descent rate during 
the major portion of the mission exceeds the maximum capability of the meter, 
this instrument should be used only in the final landing phase of the flight 
as a backup to provide flare and touchdown rates. 

Speedbrake Indicator 

The speedbrake indicator is an electromechanical instrument which in- 
dicates percent of speedbrake down from 0 to 100% in 10 percent increments. 


69 



D1 80-15082-1 


III. C. CONTROLS 
Si dearm Controller 

The sidearm controller, of the X-20 type, has motion in the fore and 
aft and lateral axes corresponding to inputs for pitch and roll, respecti vely. 
A white xnurled knob for roll trim located on top of the control handle is 
not operational and should not be used. On the left hand side of the top of 
the control handle is an operational pitch trim switch. When the trim switch 
is pushed forward, the aircraft is trimmed nose down; when the trim switch is 
pulled a r t, the aircraft is trimmed nose up. Trim is best accomplished by 
using discrete quick inputs to the trim switch with the thumb until a neutral 
flight control condition is obtained. 

Throttle/Speedbrake 

The vehicle is unpowered during the descent, approach, and landing phase 
of the mission; therefore, while the throttle handle may be moved it is inop- 
erative and does not affect flight performance. A speedbrake control switch 
is located on the throttle control handle. It is a momentary spring-loaded 
switch. When the switch is displaced backwards or aft, the speedbrakes 
come down at a constant rate of 20 % per second reducing the ve'ocity of the 
aircraft by increasing drag. When the switch is pushed forward, the speed- 
brakes are retracted. The aerodynamic characteristics and the location of 
the speedbrakes on the vehicle will cause a change in pitch tr - m when the 
speedbraces are used: the vehicle will pitch up when the speedbrakes come 

down and will pitch down when the speedbrakes come up. Therefore, when 
operating the speedbrakes the vehicle must be retrimmed in the pitch axis. 

Rudder Pedals 

Standard rudder pedals are located in the cockpit to provide yaw con- 
trol. Depressing the right rudder pedal will yaw the aircraft to the right, 
and depressing the left rudder pedal will yaw the aircraft to the left. 

Landing Gear 

A landing gear pushbutton and two colored indicator lights provide the 
displays and controls necessary to operate the landing gear. When the 
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landing gear is up, the lights are off. When the landing gear button is 
depressed., the red light will illuminate indicating that the gear are in 
transit f''om the "up and locked" position to the "down and locked" position. 
When the gear arrive in the down and locked position, the green light will 
indicate. It should be noted that when the landing gear doors cpen and the 
gear extend, a slight increase in drag will occur tending to reduce the 
velocity of the aircraft. At this time the aircraft should be retrimmed to 
maintain the desired velocity and flight path angle. 

III. D. SIMULATION CONTROLS 

The simulator operation controls are located on the left instrument 
panel. Three colored lights are associated with a three-positicn switch. 

When the switch is selected in the "Run" position, a green light will come 
on, indicating that the system is running. At the end of the mission 
after touchdown, the switch is selected to the reset position ard a red light 
will come on, indicating that the simulator is resetting itself to the 
initial start condition. The third position is a hold position which permits 
the flight to be stopped at any point during the run and restarted from that 
point. This position should not be selected by the pilot. 
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IV. MISSION DESCRIPTION 

A. NOMINAL FLIGHT PROFILE 

You have just made a nominal de-orbit reentry pass thorugh the transi- 
tion phase. You find yourself at 31,400 feet on the 135 radial, Edwards 
Simulated TACAN, 15 nautical miles (Figure 4-1). You are on instruments. 

The winds are reported calm; visibility, 15 miles. Ceiling is 10,000 feet, 
overcast. The cloud deck is solid through 35,000 feet. 

Receive confirmation from the computer room and the mode' room that the 
system is ready. Place the simulator switch into the "Run" position; the 
green light should go on. Roll right; maintain 21° of bank; pitch attitude 
about 1° below the horizon; maintain an IAS of 240 knots. During this portion 
of the flight leave the speedboards up. The flight path descert angle 
indicated by the "gamma" bar will be about 7° down. During your initial 
turn in you will pass through the ILS glideslope plane. Maintain a clean 
configuration and your airspeed at 240 knots. As you approach the 180° 
radial roll out to maintain yourself on the localizer course. At this time 
you should be approaching the 10° glideslope. As you establish yourself on 
the glideslope, go to 60% speedboards and follow the glideslope command bar 
down, using the gamma bar to assist you. Distance and bearing information to 
the TACAN will be provided digitally on the MFD. Below 10,000 feet, the MFD 
will also present a vertical cross section of your flight profile in relation 
to the runway. 

Should you drift off the localizer or glideslope, correct back to cen- 
terline as expeditiously as possible. Remember that the momentum of the 
orbiter is such that you will have to anticipate stabilization on a flight 
vector. 

Above 6,000 feet your primary reference will probably be instruments 
with a cross check through the VFR display. The extension of a thin dark 
shadow area out from the runway indicates runway center line. Below 6,000 
feet, you 7 ' flight profile should be based primarily on the external VFR view 
with cross check on instruments. However, you should also become familiar 
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Figure 4-1: F LIGHT SCHEMATIC - EDWARDS AFB (SIM) SIMULATION APPROACH, H-33 ORBITFR 
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with the appearance of the approach above 6,000 feet, because several 
approaches will be conducted with a terminal navigation failure in which 
the only safe reference is the VFR display. 

At 2,200 feet, your radar altimeter will become operational. Use your 
radar altimeter to establish the 750-foot gear extension and flare point. 

At 750 feet, you should reduce your descent rate to follow a 3° glideslope. 

Use the flight-path-angle (gamma) reference bar to establish this glideslope 
of 3°. Hold the 3° flight path angle until you cross the threshold; at this 
point you should flare. Transition to a 300 fpm descent rate so as to touch 
down 2,000 feet down the runway. After you touchdown (indicated by the touch- 
down beep), return the simulation control to the RESET position and await 
instruction for the beginning of the next flight. 

Should you experience an emergency or flight control malfunction in the 
flight, carry out the standard emergency procedures that you have learned in 
training. Remember safe control and flight of the orbiter is paramount. 

IV. B. SUMMARY OF KEY FLIGHT CONTROL POINTS 

1. Initial Cond ition 

A1 ti tude 
Ai rspeed 
Course 
Posi tion 

2. Turn-In 

Set up 21° bank to the right, maintain 240 knots. 

Flight path angle approximately 7°. 

Pass through glideslope plane. Do not establish on the 
glideslope until approximately straignt-in. 


31,400 feet 
240 knots 
225 degrees 

135 radial, Edwards TACAN, 15 nautical miles 
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3. Line Up on ILS Glideslope and Localizer 

a) Before TACAN: Establish yourself on the 180 radial, 

Course 360 to TACAN. When ILS glideslope intercepted, 
drop speedboards to 60%, nose down to maintain 240 knots. 

b) After TACAN: Maintain the orbiter on the ILS localizer, 

Course 360 from the TACAN. Establish a 10° glideslope, 
descent on the ILS glideslope. 


4. Final Approach 

10° glideslope 
240 knots 
Course 360, (000) 

Cross check with VFR display below 10,000 feet 
Concentrate on VFR line up below 6,000 feet 


5. Flare 

Altitude 750 feet 
Gear Down 

Rotate smoothly to 5° nose up 
Establish 3° glideslope 


6. Threshold 

Altitude 200 feet 
Airspeed 180-190 knots 

Rotate smoothly to 10° nose up, establish a 1° glideslope 

7. Touchdown 

On center line 

2,000 ft. marker 

4 to 6 fps descent rate 

Airspeed approximately 160-170 knots 
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IV. C. PHOTOS OF DISPLAYS AT KEY POINTS 

Eight figures are presented (Figures 4-2 through 4-9) to familiarize 
you with the appearance of the displays at various points throughout the 
approach. The figures are listed below: 


Fig. 

4-2. 

Initial Position; 31,400 feet altitude 


Fig. 

4-3. 

Inbound to TACAN: Approaching 180° Radial; 20,000 

Altitude. 

ft 

Fig. 

4-4. 

At the TACAN: Altitude 12,000 feet; on Course, slightly 

above G1 ideslope. 

Fig. 

4-5. 

Inbound Final; Altitude 9,800 feet; just breaking 
slightly left; on G1 ideslope; Speedboards 70%. 

VFR; 

Fig. 

4-6. 

Inbound Final; Altitude 5,600 feet; on Course; on 
G1 ideslope. 


Fig. 

4-7. 

Landing Transition; Altitude 750 feet; Gear Down; 
to 5° Nose Up. 

FI are 

Fig. 

4-8. 

Over Threshold; Altitude 100 feet; Rotate. 


Fig. 

4-9. 

At Touchdown; Altitude 30 feet; 200 fpm Descert Rate. 
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Figure 4-2: INITIAL POSITION; AL TITUDE 31,400 FEET 
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Figure 4-3: INBOUND TO TACAN: APPROACH 180° RADIAL; 20,000 FT ALTITUDE 
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Figure 4-4: A T THE TACAN: ALTITUDE 12,000 FEET; ON COURSE, SLIGHTL Y ABOVE GLIDES LOPE 
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Figure 4-5: INBOUND FINAL; ALTITUDE 9,800 FEET; JUST BREAKING VFR;SL!GHTL Y LEFT; ON GLIDESLOPE; SPEEDBOARD 70% 
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Figure 4-6: INBOUND FINAL; ALTITUDE 5,600 FEET; ON COURSE; ON GLIDESLOPE 



I 
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Figure 4-7: LANDING TRANSITION; ALTITUDE 750 FEET; GEAR DOWN; FLARE TO 5° NOSE UP 
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Figure 4-8: OVER THRESHOLD; ALTITUDE 100 FEET; ROTA TE 
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Figure 4-9: A T TOUCHDOWN; A L TITUDE 30 FEET; 200 FPM DESCENT HA TE 
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V. EMERGENCY PROCEDURES 

A. GENERAL 

Emergency systems and procedures have been established for the simu- 
lator to provide more realistic subsystems management requirements during the 
mission and to permit the testing of flight performance during periods of 
stress or increased task loading. 

You will be expected to learn the proper procedures to correct system 
failures and to handle them expeditiously. Remember, safe control of your 
vehicle is most important; correct, rather than hasty, actuation of system 
switches is essential. However, you will be tested on the speed as well as 
the correctness of your responses, so a concentrated subsystems training 
program will be required. 

B. COMPLETE SUBSYSTEMS SCAN 

The major subsystems of the vehicle are constantly monitored for nor- 
mal operation by a Master Caution and Advisory system. When a failure is de- 
tected by the system, the master failure warning light will illuminate and 
the onboard system computer will identify the suspect system by illuminating 
a legend on the Subsystem Monitoring Panels (Fig. 5-1). The master warning 
light will remain illuminated until the pilot acknowledges the failure by 
depressing the Subsystem Failure Switch Light on the subsystem Failure 
Advisory Panel, or selecting a secondary SAS subsystem on the Stability 
Augmentation System (SAS) Monitor and Control Panel. 

A computer-controlled troubleshooting system is also provided. When 
properly actuated, this system starts a computer scan that attempts to isolate 
the faulty component or network and reroute the signals and/or make use of 
alternate supply sources. This scan normally takes approximately 15 seconds. 
The system is started by depressing the identified system switch light (iden- 
tified by its flashing light) and then depressing the Fail-Scan switch. Both 
switches will light steady during the scan. At completion, the Fail-Scan 
light will go out. If the faulty subsystem can be corrected, the failure in- 
formation light will also go out. If no correction is possible, the sub- 
system "ailure light will remain on. 
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C. STABILITY AUGMENTATION SYSTEM (SAS) 

The H-33 orbiter simulator is fully aerodynamic and includes a stability 
augmentation system (SAS) in all three axes--pitch, roll, and yaw. A Selector 
panel in the cockpit permits manual deactivation or alternate mode switching 
for any cr all of the axes. 

The SAS system has three redundant sensor/signal comparing modules for 
each axis: SAS 1, SAS 2, and SAS Alternate. Each module is divided into 

redundant primary and secondary control units, composed of sensors and signal 
conditioning equipment, which produce control signals dependent upon vehicle 
performance, environmental conditions, and pilot input. The outputs of the 
primary and secondary control units are balanced by a comparator which then 
transmits summed control signals to the flight control system. A failure is 
sensed and indicated if either control unit operating voltage is out of 
tolerance. 

Each of the SAS control units operates under a normal system voltage 
of 28 + 1.5 vdc. If the SAS Failure light illuminates, it indicates that 
primary or secondary unit supply voltage exceeds these normal limits or that 
an internal failure of a system component has occurred. If a failure occurs, 
the next SAS system in line should be selected and the operating voltages of 
the selected mode checked twice--once for the primary unit and once for the 
secondary unit. The normal mode progression sequence for SAS operations is: 

(1) SAS, (2) SAS 2, (3) SAS Alt, and (4) SAS Off. 

The SAS system is not completely self switching. All the mode (SAS 1, 
SAS 2, SAS Alt, SAS Off) switching, and the primary and seconde.ry unit 
operating voltage checks must be initiated manually. When performing the 
unit voltage checks, however, the digital voltmeter is automatically 
sequenced from the primary to the secondary unit. If more than two voltage 
checks a^e made for any one mode, or a SAS mode is selected out of sequence, 
an overload light will illuminate. The system must be reset, and voltage 
checking restarted on the SAS 2 mode. 
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A logic diagram of the 
in Figure 5-2. The logic is 
failure procedures check-off 


procedures for a SAS Pitch failure is presented 
the same for roll and yaw failures. A SAS 
list is presented in Table 5-1. 
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Figure 5-2: SAS PITCH FAILURE PROCEDURE CONTROL LOGIC DIAGRAM 
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TABLE 5-1. SAS FAILURE PROCEDURES CHECK LIST 


1. Failure Occurs (Pitch mode chosen as example) 

a) "Master Failure" lights blinks. 

b) "SAS Fail-Pitch" light blinks. 

c) "Pitch-1" light blinks. 


2. Push "Pitch-2" 

a) "Master Failure" light goes out. 

b) "SAS Fail-Pitch" light stops blinking (remains on). 

c) "Pitch-1" goes out. 

d) "Pitch-2" goes on. 


3. Push and hold in "System Check" switch. 

a) Optimum system value plus tolerance limits 

appear in lower meter window. 

b) System value appears in upper meter window: 

1) If meter reads within tolerance, release 
and re-depress switch to check the secondary 
system. If secondary is also normal, continue 
flight in selected mode (i.e., Pitch-2). 

2) If meter reads to high or too low (out of 
tolerance) at any point in the check sequence, 
immediately go to the next SAS mode (i.e., 
Alternate) in the system, recheck, and continue. 

3) If meter reads zero at any point in the check, 
immediately go to system off - (i.e., "Pitcn Off"; 
flight continues with pitch SAS inoperative.) 


4. If overload reset light comes on, push Overload Reset switch. 

This resets system to Mode-2, ready for voltage checks; continue. 


Summary : Flight normally begins in Pitch-1. If system mal funci tons , 

Pitch-2 is selected. If Pitch-2 is out of tolerance (other than zero), 
Pitch-Alternate is selected. If Pitch-Alternate is out, Pitch-Off is 
selected. 
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